This Week in AI was WILD: Grok 2, Claude, SearchGPT, AgentQ and AI Scientist You Can’t Afford to…

🌈 Abstract

The article covers the latest developments in AI, including Anthropic's prompt caching feature for Claude, the impressive performance of Grok 2 model, the rise of search engines like SearchGPT, the challenges faced by AI agents in navigating complex web tasks, and the groundbreaking AI Scientist tool from Sakana AI. It also discusses the hype around AI and the need for skepticism, as well as a failed demo by Google.

🙋 Q&A

[01] Anthropic's Prompt Caching

1. What is prompt caching and how does it work?

• Prompt caching allows users to keep a summary of their entire conversation history in the cache, avoiding the need to re-send the same context information with every API call.
• This can provide a significant boost in latency and cost savings, especially for scenarios involving large document processing, detailed instruction sets, or reusing the same context over multiple requests.
• The cache has a 5-minute lifetime but refreshes with each use, and users can set up to four cache breakpoints to quickly access the exact context needed.

2. How does prompt caching impact pricing and performance?

• Writing to the cache costs about 25% more than the base input token price, but using the cached content is much cheaper, costing only 10% of the normal price.
• This can result in substantial cost savings, especially for users working with large amounts of data.
• Notion has already implemented prompt caching for their Notion AI, optimizing their operations to deliver faster, cheaper, and better experiences to their users.

[02] Grok 2 Model

1. What is the Grok 2 model, and why is it generating excitement?

• Grok 2 is a powerful AI model developed by Elon Musk's AI venture, x.ai.
• It is outperforming models like Claude 3.5 Sonnet and GPT-4-Turbo on the LMSYS leaderboard, particularly in the areas of logic and reasoning.
• Grok 2 will be available through an enterprise API later this month, and its impressive performance is seen as a challenge to Google's AI efforts.

2. What are the key highlights of the Grok 2 model?

• Grok 2 is excelling in logic and reasoning tasks, outperforming other prominent models.
• It is seen as a threat to Google's AI dominance, as the talent behind Google's groundbreaking work has been leaving the company for more lucrative opportunities elsewhere.
• There is speculation that the model may need to be rebranded, as the name "Grok" started as a bit of a joke but now reflects its growing power and significance.

[03] SearchGPT and the Changing Landscape of Search

1. How are SearchGPT and Perplexity disrupting the traditional search engine model?

• SearchGPT and Perplexity are providing a new way of navigating the web, offering personalized and relevant results without the clutter of ads and endless blue links.
• They are blending conversational AI with real-time web information, allowing users to get clear, concise answers to their queries, backed by relevant sources and the latest media.
• This is seen as a significant challenge to Google's dominance in the search engine market.

2. What are the key benefits of using SearchGPT and Perplexity over traditional search engines?

• Users can get the information they need quickly and efficiently, without having to sift through irrelevant results.
• The search experience is more personalized and tailored to the user's needs, providing a better overall experience.
• The technology is seen as a crucial development in how people navigate the internet, with the potential to become a key way of accessing information in the future.

[04] Challenges Faced by AI Agents in Web Navigation

1. What are the limitations of current AI agents when it comes to navigating complex, dynamic web tasks?

• AI agents trained on static datasets struggle with the chaotic and unpredictable nature of the internet, often making small mistakes that snowball into bigger issues.
• They are not well-equipped to handle the dynamic, real-world interactions required for tasks like booking a table through a website.

2. How does the AgentQ model address these limitations?

• AgentQ combines guided Monte Carlo Tree Search (MCTS) and AI self-critique to improve its ability to navigate the web, make decisions, and correct itself when things go wrong.
• In a booking experiment on OpenTable, AgentQ was able to improve the success rate of the LLaMa-3 model from 18.6% to 81.7% after just one day of learning, and up to 95.4% with online search enabled.
• This represents a significant leap in the capabilities of AI agents when it comes to handling complex, real-world web tasks.

[05] Sakana AI's AI Scientist

1. What is the AI Scientist tool developed by Sakana AI, and how is it different from other AI tools?

• The AI Scientist is a groundbreaking tool that can autonomously generate hypotheses, run experiments, and write research papers, all for just $15 per paper.
• Unlike other AI tools that simply follow commands or regurgitate information, the AI Scientist is capable of discovering new knowledge on its own, pushing the boundaries of what is possible with AI.

2. What are the potential implications of the AI Scientist tool?

• The AI Scientist could democratize research and speed up scientific progress in ways that were previously unimaginable, by making the research process more accessible and cost-effective.
• The ability of the AI Scientist to autonomously generate and validate new knowledge has the potential to revolutionize how scientific research is conducted.

[06] Google's Struggles and the Need for Skepticism

1. What happened during the live demo of Google's features, and what does it reveal about the company's challenges?

• The live demo of a feature that could check a user's calendar after snapping a photo of a concert poster failed, not once but twice, highlighting the gap between Google's promises and the actual functionality of their products.
• This incident raises questions about the company's focus on data and analytics to "improve features and functionality," as the demo suggests that these efforts may not be translating into meaningful improvements for users.

2. How does the article suggest dealing with the hype around AI?

• The article cautions against the growing hype around AI, noting that it can attract "grifters" trying to build a following or make a quick buck.
• It suggests that while the hype may lead to temporary skepticism, people will likely move on to the next hype master soon enough, emphasizing the need for a more critical and discerning approach to AI developments.